According to a conventional augmented reality technology, a virtual object is additively displayed in correspondence with a marker in a real space imaged with a camera.
However, when processing on position and attitude is performed on the basis of the marker in the real space according to the conventional augmented realty technology, the processing relies on a single marker used as the reference, and thus the flexibility of the processing is low.
An example of an information processing program according to this disclosure is an image processing system including a component marked with a feature whose position and attitude relative to an imaging device can be specified by being imaged by the imaging device, and an image processing apparatus connected to the imaging device. The image processing apparatus includes: captured image acquisition means for acquiring a captured image captured by the imaging device; feature detection means for detecting the feature from the captured image; reference acquisition means for acquiring a position/attitude reference based on the detected feature, the reference being used as a reference for indicating a position and attitude in a space imaged by the imaging device; and relative relation information acquisition means for acquiring, based on the captured image from which the feature is detected in plurality, relative relation information indicating a relative relation in position and attitude among the plurality of position/attitude references acquired for the respective features.
In this disclosure, the feature whose position and attitude relative to the imaging device can be specified is, specifically, a so-called AR marker or a code such as a two-dimensional bar code. Such a feature may be applied to a component such as a card.
The position/attitude reference is a reference indicating a position and attitude in an imaged space. The position/attitude reference may be a coordinate system in which the origin and three axes are defined according to the position and attitude of the marker, or position/attitude information of the marker relative to the imaging device.
The relative relation information is information indicating a relative relation in position and attitude among the plurality of position/attitude references. The relative relation information may be a conversion matrix used for performing coordinate conversion between the coordinate systems as the position/attitude references.
By having the means describes above, the image processing system according to this disclosure is able to acquire relative relation information indicating a relative relation in position and attitude among the plurality of the position/attitude references.
This makes it possible to improve the flexibility of processing when the processing is performed for position and attitude while using a feature such as a so-called AR marker as the reference.
The image processing apparatus may further include estimation means for estimating, based on the relative relation information and one of the plurality of position/attitude references related to the relative relation information, another one of the position/attitude references.
The provision of this estimation means makes it possible to obtain a position/attitude reference by estimation from another position/attitude reference, whereby the processing flexibility can be improved.
For example, in order to perform processing based on a plurality of markers according to a conventional technique, these markers are imaged such that all of them are within the imaging range of an imaging device. Therefore, these markers need be simultaneously positioned within the imaging range of the imaging device, which narrows the moveable range of the imaging device and causes inconvenience to a user. However, the image processing apparatus according to this disclosure is provided with estimation means, which eliminates the need of positioning all the markers within the imaging range, and hence the flexibility in movement of the imaging device can be improved.
The image processing apparatus may further include storage means for storing the relative relation information.
The image processing apparatus may further includes: determination means for determining whether or not the relative relation information acquired by the relative relation information acquisition means is relative relation information corresponding to the relative relation information stored in the storage means; and update means for updating the stored relative relation information using the acquired relative relation information, when it is determined by the determination means that the acquired relative relation information is relative relation information corresponding to the stored relative relation information.
The provision of such update means makes it possible, if the position where a feature is arranged is changed or an error occurs in calculation of position and attitude of the feature, to keep the stored relative relation information updated and to enable it to follow the movement of the feature.
The image processing apparatus may further include addition means for storing the acquired relative relation information in the storage means when it is not determined by the determination means that the acquired relative relation information is relative relation information corresponding to the stored relative relation information.
The provision of such addition means makes it possible, even if a feature added, to keep the stored relative relation information updated and to enable it to follow the addition of the feature.
The image processing apparatus may further include deletion means for deleting the relative relation information satisfying a predetermined condition from the storage means.
The provision of such deletion means makes it possible to keep the stored relative relation information updated, even if the feature is removed, or the use of a feature as a position/attitude reference is halted during the course of processing.
The image processing apparatus may further include rendering means for rendering a virtual space image by arranging a virtual camera in a virtual space, arranging, in the virtual space, a virtual object whose position and attitude are determined according to the position/attitude reference, and generating an image of the virtual space as viewed from the virtual camera.
The image processing apparatus may further include display control means for generating a composite image by overlapping the image of the virtual space with the captured image and causing a display device to display the composite image.
The image processing apparatus may further include processing means for performing predetermined processing so that time in the virtual space advances with passage of real time.
This disclosure can be considered as a method implemented by a computer, or a program executed by a computer.
This disclosure may be a recording medium on which such a program is recorded and which is readable by a computer or the like. The recording medium readable by a computer or the like herein means a recording medium which stores information such as data or a program by means of an electrical, magnetic, optical, mechanic, or chemical effect such that the stored information can be read by a computer or the like.